Looking Under the Hood : Tools for Diagnosing your Question Answering Engine
نویسندگان
چکیده
In this paper we analyze two question answering tasks : the TREC-8 question answering task and a set of reading comprehension exams. First, we show that Q/A systems perform better when there are multiple answer opportunities per question. Next, we analyze common approaches to two subproblems: term overlap for answer sentence identification, and answer typing for short answer extraction. We present general tools for analyzing the strengths and limitations of techniques for these subproblems. Our results quantify the limitations of both term overlap and answer typing to distinguish between competing answer candidates.
منابع مشابه
Looking Under the Hood: Tools for Diagnosing Your Question Answering Engine1
In this paper we analyze two question answering tasks : the TREC-8 question answering task and a set of reading comprehension exams. First, we show that Q/A systems perform better when there are multiple answer opportunities per question. Next, we analyze common approaches to two subproblems: term overlap for answer sentence identification, and answer typing for short answer extraction. We pres...
متن کاملOptimizing question answering systems by Accelerated Particle Swarm Optimization (APSO)
One of the most important research areas in natural language processing is Question Answering Systems (QASs). Existing search engines, with Google at the top, have many remarkable capabilities. But there is a basic limitation (search engines do not have deduction capability), a capability which a QAS is expected to have. In this perspective, a search engine may be viewed as a semi-mechanized QA...
متن کاملLooking under the Hood of Stochastic Machine Learning Algorithms for Parts of Speech Tagging
The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the National Science Foundation, or the U.S. government. We are grateful to Alex Rudnicky from CMU for providing the training data to us and to Yifen Huang, CMU, for discussing the project with us. Abstract A variety o...
متن کاملDefinitional Question-Answering Using Trainable Text Classifiers
Automatic question answering (QA) has gained increasing interest in the last few years. Question-Answering systems return an answer rather than a document. Definitional questions are questions such as Who is Alexander Hamilton? or what are fractals? Looking at logs of web search engines definitional questions occur quite frequently, suggesting it is an important type of questions. Analysis of p...
متن کاملVisual Question Answering Using Various Methods
This project tries to apply deep learning tools to enable computer answering question by looking at images. In this project, the visual question answering dataset[1] is introduced. This dataset consists of 204,721 real images, 614,164 question and 50,000 abstract scenes, 150,000 questions. Various methods are reproduced. The analysis on different models are presented.
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره cs.CL/0107006 شماره
صفحات -
تاریخ انتشار 2001